Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

weaver: bump version to 5.1.0 + debug notebook output #449

Merged
merged 2 commits into from
May 7, 2024
Merged

weaver: bump version to 5.1.0 + debug notebook output #449

merged 2 commits into from
May 7, 2024

Conversation

fmigneault
Copy link
Collaborator

Overview

Debugging cause of diff in final output of https://github.com/Ouranosinc/pavics-sdi/blob/master/docs/source/notebook-components/weaver_example.ipynb

Built on top of #439

Changes

Non-breaking changes

  • New component version Weaver: 5.1.1
  • Fixes to Weaver WSP provider registration retry logic.

Related Issue / Discussion

@github-actions github-actions bot added component/weaver Related to https://github.com/crim-ca/weaver documentation Improvements or additions to documentation feature/WPS Feature or service related to Web Processing Service labels Apr 23, 2024
@fmigneault fmigneault changed the title weaver: bump version to 5.1.0 weaver: bump version to 5.1.0 + debug notebook output Apr 23, 2024
@crim-jenkins-bot
Copy link
Collaborator

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2612/
Result ✅ SUCCESS

BIRDHOUSE_DEPLOY_BRANCH : fix-weaver-retry
DACCS_IAC_BRANCH : master
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1586/

NOTEBOOK TEST RESULTS 
    
[2024-04-23T15:06:24.595Z] ============================= test session starts ==============================
[2024-04-23T15:06:24.595Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-04-23T15:06:24.595Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-04-23T15:06:24.595Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-04-23T15:06:24.595Z] collected 259 items
[2024-04-23T15:06:24.595Z] 
[2024-04-23T15:06:35.804Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-04-23T15:07:14.762Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-04-23T15:07:18.538Z] notebooks-auth/test_thredds.ipynb ...........                            [ 15%]
[2024-04-23T15:08:05.314Z] pavics-sdi-master/docs/source/notebooks/CaSR_basic.ipynb ......          [ 17%]
[2024-04-23T15:08:13.634Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 20%]
[2024-04-23T15:08:23.346Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 22%]
[2024-04-23T15:08:29.736Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb ........       [ 25%]
[2024-04-23T15:15:50.434Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 30%]
[2024-04-23T15:15:52.349Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 30%]
[2024-04-23T15:16:00.647Z] ...............                                                          [ 36%]
[2024-04-23T15:16:09.714Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 38%]
[2024-04-23T15:16:16.556Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 40%]
[2024-04-23T15:16:32.450Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 42%]
[2024-04-23T15:16:39.438Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 45%]
[2024-04-23T15:16:44.202Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 47%]
[2024-04-23T15:22:09.482Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 53%]
[2024-04-23T15:23:21.069Z] .............                                                            [ 58%]
[2024-04-23T15:23:23.512Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 59%]
[2024-04-23T15:23:26.038Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 62%]
[2024-04-23T15:23:43.053Z] .................                                                        [ 69%]
[2024-04-23T15:23:50.001Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 71%]
[2024-04-23T15:24:01.069Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-04-23T15:24:10.066Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 78%]
[2024-04-23T15:24:11.451Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 78%]
[2024-04-23T15:24:14.257Z] ......                                                                   [ 81%]
[2024-04-23T15:24:20.840Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 81%]
[2024-04-23T15:24:35.354Z] .............                                                            [ 86%]
[2024-04-23T15:24:45.364Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 86%]
[2024-04-23T15:25:23.415Z] ....s.                                                                   [ 89%]
[2024-04-23T15:25:31.570Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-04-23T15:25:44.845Z] ..                                                                       [ 90%]
[2024-04-23T15:25:59.761Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 90%]
[2024-04-23T15:26:23.089Z] ......                                                                   [ 93%]
[2024-04-23T15:26:24.730Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-04-23T15:28:59.156Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-04-23T15:28:59.156Z] 
[2024-04-23T15:28:59.156Z] ================= 258 passed, 1 skipped in 1348.26s (0:22:28) ==================

@fmigneault
Copy link
Collaborator Author

@mishaschwartz
So, I still have no idea what could be the cause of the hummingbird output sporadically differing from the notebook cell, but all tests are now succeeding.

@fmigneault fmigneault marked this pull request as ready for review April 23, 2024 15:31
@fmigneault fmigneault self-assigned this Apr 23, 2024
@fmigneault
Copy link
Collaborator Author

Hold up! Weaver notebook was not actually tested during that run... weird.
🕵️ investigating

tlvu
tlvu approved these changes Apr 23, 2024
View reviewed changes
Copy link
Collaborator

@tlvu tlvu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I defer to Misha that knows more about Weaver than I do.

@fmigneault
Copy link
Collaborator Author

Found the cause. Due to invalid sourcing of env.local over capturing outputs to pass to TEST_PAVICS_SDI_WEAVER.

PR in progress to fix (https://gitlab.com/crim.ca/clients/daccs/daccs-e2e-jenkins/-/merge_requests/33). Will rerun once integrated.

@crim-jenkins-bot
Copy link
Collaborator

E2E Test Results

DACCS-iac Pipeline Results

Build URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2624/
Result ❌ FAILURE

BIRDHOUSE_DEPLOY_BRANCH : fix-weaver-retry
DACCS_IAC_BRANCH : master
DACCS_CONFIGS_BRANCH : master
PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master
PAVICS_SDI_BRANCH : master

DESTROY_INFRA_ON_EXIT : true
PAVICS_HOST : https://host-140-154.rdext.crim.ca

PAVICS-e2e-workflow-tests Pipeline Results

Tests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1590/

NOTEBOOK TEST RESULTS 
    
[2024-05-01T14:46:23.376Z] ============================= test session starts ==============================
[2024-05-01T14:46:23.376Z] platform linux -- Python 3.9.16, pytest-7.3.1, pluggy-1.0.0
[2024-05-01T14:46:23.376Z] rootdir: /home/jenkins/agent/workspace/PAVICS-e2e-workflow-tests_master
[2024-05-01T14:46:23.376Z] plugins: anyio-3.6.1, dash-2.10.0, nbval-0.9.6, tornasync-0.6.0.post2, xdist-3.3.1
[2024-05-01T14:46:23.376Z] collected 269 items
[2024-05-01T14:46:23.376Z] 
[2024-05-01T14:46:33.689Z] notebooks-auth/geoserver.ipynb ..................                        [  6%]
[2024-05-01T14:47:02.490Z] notebooks-auth/test_cowbird_jupyter.ipynb ..........                     [ 10%]
[2024-05-01T14:47:06.449Z] notebooks-auth/test_thredds.ipynb ...........                            [ 14%]
[2024-05-01T14:47:54.295Z] pavics-sdi-master/docs/source/notebooks/CaSR_basic.ipynb ......          [ 16%]
[2024-05-01T14:48:03.280Z] pavics-sdi-master/docs/source/notebooks/WCS_example.ipynb .......        [ 19%]
[2024-05-01T14:48:13.350Z] pavics-sdi-master/docs/source/notebooks/WFS_example.ipynb ......         [ 21%]
[2024-05-01T14:48:17.210Z] pavics-sdi-master/docs/source/notebooks/WMS_example.ipynb .FFFFFFF       [ 24%]
[2024-05-01T14:55:55.138Z] pavics-sdi-master/docs/source/notebooks/climex.ipynb ............        [ 28%]
[2024-05-01T14:55:55.138Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-climate-stations.ipynb . [ 29%]
[2024-05-01T14:55:55.928Z] ...............                                                          [ 34%]
[2024-05-01T14:56:04.869Z] pavics-sdi-master/docs/source/notebooks/eccc-geoapi-xclim.ipynb .....    [ 36%]
[2024-05-01T14:56:11.980Z] pavics-sdi-master/docs/source/notebooks/esgf-dap.ipynb ......            [ 39%]
[2024-05-01T14:56:28.381Z] pavics-sdi-master/docs/source/notebooks/forecasts.ipynb ......           [ 41%]
[2024-05-01T14:56:33.774Z] pavics-sdi-master/docs/source/notebooks/opendap.ipynb .......            [ 43%]
[2024-05-01T14:56:38.642Z] pavics-sdi-master/docs/source/notebooks/pavics_thredds.ipynb .....       [ 45%]
[2024-05-01T15:00:06.074Z] pavics-sdi-master/docs/source/notebooks/regridding.ipynb ............... [ 51%]
[2024-05-01T15:01:18.809Z] .............                                                            [ 56%]
[2024-05-01T15:01:21.567Z] pavics-sdi-master/docs/source/notebooks/rendering.ipynb ....             [ 57%]
[2024-05-01T15:01:23.886Z] pavics-sdi-master/docs/source/notebooks/subset-user-input.ipynb ........ [ 60%]
[2024-05-01T15:01:41.388Z] .................                                                        [ 66%]
[2024-05-01T15:01:48.058Z] pavics-sdi-master/docs/source/notebooks/subsetting.ipynb ......          [ 69%]
[2024-05-01T15:01:49.446Z] pavics-sdi-master/docs/source/notebook-components/weaver_example.ipynb . [ 69%]
[2024-05-01T15:02:07.217Z] ........F                                                                [ 72%]
[2024-05-01T15:02:16.590Z] finch-master/docs/source/notebooks/dap_subset.ipynb ...........          [ 76%]
[2024-05-01T15:02:25.592Z] finch-master/docs/source/notebooks/finch-usage.ipynb ......              [ 79%]
[2024-05-01T15:02:26.971Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-1DataAccess.ipynb . [ 79%]
[2024-05-01T15:02:30.048Z] ......                                                                   [ 81%]
[2024-05-01T15:02:40.051Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-2Subsetting.ipynb . [ 82%]
[2024-05-01T15:02:54.929Z] .............                                                            [ 86%]
[2024-05-01T15:03:04.959Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-3Climate-Indicators.ipynb . [ 87%]
[2024-05-01T15:03:43.802Z] ....s.                                                                   [ 89%]
[2024-05-01T15:03:51.942Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-4Ensembles.ipynb . [ 89%]
[2024-05-01T15:04:04.756Z] ..                                                                       [ 90%]
[2024-05-01T15:04:19.672Z] PAVICS-landing-master/content/notebooks/climate_indicators/PAVICStutorial_ClimateDataAnalysis-5Visualization.ipynb . [ 91%]
[2024-05-01T15:04:41.791Z] ......                                                                   [ 93%]
[2024-05-01T15:04:44.551Z] notebooks/hummingbird.ipynb ............                                 [ 97%]
[2024-05-01T15:07:18.723Z] notebooks/stress-tests.ipynb ......                                      [100%]
[2024-05-01T15:07:18.723Z] 
[2024-05-01T15:07:18.723Z] =================================== FAILURES ===================================

@fmigneault fmigneault merged commit b6dc1e8 into bump-weaver-5.1.0 May 7, 2024
4 of 5 checks passed
@fmigneault fmigneault deleted the fix-weaver-retry branch May 7, 2024 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tlvu tlvu tlvu approved these changes

@mishaschwartz mishaschwartz Awaiting requested review from mishaschwartz mishaschwartz is a code owner

Assignees

@fmigneault fmigneault

Labels
component/weaver Related to https://github.com/crim-ca/weaver documentation Improvements or additions to documentation feature/WPS Feature or service related to Web Processing Service
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@fmigneault @crim-jenkins-bot @tlvu